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Background: Through application of the verbal autopsy (VA) approach, trained fieldworkers collect information 
about the probable cause of death (COD) by using a standardized questionnaire to interview family members who 
were present at the time of death. The physician-certified VA (PCVA), an independent review of this questionnaire 
data by up to three physicians trained in VA coding, is currently recommended by the World Health Organization 
(WHO) and is widely used in the INDEPTH Network. Even given its appropriateness in these contexts, a large 
percentage of causes of death assigned by VAs remains undetermined. As physicians often do not agree upon a 
final COD classification, there remains substantial room to improve the standard VA method, potentially leading to 
a reduction in physician discordance in COD coding. 

Methods: We present an extension of the current method of PCVA and compare it to the standard WHO- 
recommended procedure. We used VA data collected in the Nouna Health and Demographic Surveillance Site 
(NHDSS) between 2009 and 2010 using a locally-adapted version of an INDEPTH standard verbal autopsy 
questionnaire. Until 2009, physicians in the NHDSS followed the WHO method (Method 1). As an extension of 
Method 1, starting in 2010, the use of a panel of physicians was added to the coding process in the case where a 
third physician's final conclusions resulted in an undetermined COD (Method 2). Two independent samples of VA 
questionnaires were compared for the year 2009 (using Method 1) and the year 2010 (using Method 2). 

Results: The WHO-recommended method used for 2009 yielded a high level of undetermined CODs, where the 
final coding was "undetermined" in 50.8% of all questionnaires due to disagreement among participating 
physicians (Method 1). By introducing a panel of physicians in 2010 for cases where the principal physicians 
disagreed on the cause of death, the revised method significantly reduced the proportion of undetermined CODs 
to 1 .5% (Method 2). 

Conclusions: As the extended method of PCVA significantly improved the accuracy of the VA procedure, we 
suggest the adoption of this method for those countries where alternatives like computer-based VA coding are not 
available. Based on the results of our study, further research should be pursued. 
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Background 

Verbal autopsy (VA) is a technique used to determine 
the cause of death by asking caregivers, friends, or 
family members about signs and symptoms exhibited 
by the deceased in the period before death. It is usually 
done by trained fieldworkers using a standardized 
questionnaire that collects details on signs, symptoms, 
complaints, and any medical history or events prior to 
death [1]. 

The World Health Organization (WHO) recommends 
the use of verbal autopsy to measure specific causes of 
death [2,3]. The purpose of verbal autopsy is to describe 
the causes of death at the community or population 
level where limited or no vital registration is completed 
with medical certificates. Indeed, medically certified 
cause of death data are available for less than one-third 
of the more than 57 million deaths occurring worldwide 
annually. The majority of deaths lacking such data are 
from developing countries [4]. Information about cause 
of death is essential for public health planning, priority 
setting, monitoring, and evaluation, but the collection of 
such information in countries with incomplete or no 
vital registration systems remains a substantial challenge 
[5]. Reliable data on cause-specific mortality is also 
needed by countries to keep track of progress toward 
the Millennium Development Goals [3,6]. 

The use of physician-certified verbal autopsies (PCVA) 
is common in the majority of developing countries, as 
well as for Health and Demographic Surveillance Sites 
(HDSS) that are members of International Network for 
the Demographic Evaluation of Populations and Their 
Health in Developing Countries (INDEPTH) [6]. Within 
the INDEPTH Network, 36 HDSS in 20 countries regu- 
larly use VA to assess cause of death [6]. However, the 
data collection tools are not yet harmonized, which has 
led to substantial variability in the coding process across 
sites [7,8]. Recently, there have been several attempts to 
introduce alternative methods such as a computer-based 
verbal autopsy coding method (InterVA) to replace the 
PCVA approach [6]. This probability-based method was 
tested in several settings [9-11]. However, the results 
still show some discrepancies in comparison to PCVA 
results [12]. Few studies are available on the use of dif- 
ferent physician coding methods that produce better 
results. In contrast, the study by Joshi and colleagues 
comparing results involving multiple coders versus one 
single coder suggest that advantages attained from the 
multiple-coding system remain limited [13]. However, in 
this study, the approaches for cause of death assignment 
used either a panel of expert physicians or involved two 
or more physician coders who independently reviewed 
the data to arrive at a final diagnosis [14,15]. The 
method proposed in this paper was tested with the 



intention to build on former approaches, such as those 
presented in the study by Joshi et al [13]. 

In this study we compare the usual WHO-recom- 
mended PCVA procedure with a locally-adapted method 
that incorporates the use of a panel of physicians after a 
discrepancy among three physician coders arises. 

Methods 

Study area 

The Nouna Health and Demographic Surveillance Site 
(NHDSS) has existed since 1992 and is in the rural wes- 
tern part of Burkina Faso (Figure 1). It currently covers 
58 villages and one semi-urban town and covers a popu- 
lation of about 85,000 inhabitants. The NHDSS is part 
of Kossi province, which consists primarily of a rural 
population of multi-ethnic groups. The predominant 
activity is subsistence farming and cattle keeping. The 
region is a dry orchard savannah and has a sub-Sahelian 
climate, which is characterized by a hot climate with 
short rainy season lasting from June to September with 
rainfall varying between 400 to 1000 millimeters. The 
vegetation is mainly scattered short trees. The mean 
temperature varies from 26°C to 34°C, often reaching 
40°C in April, the hottest period [11]. 

The NHDSS is a member of the INDEPTH Network, 
a global network of HDSSs with the aim of conducting 
longitudinal health and demographic evaluation of 
populations in low- and middle-income countries [16]. 
The health facilities within the NHDSS consist of one 
secondary care facility (the district hospital) and 14 pri- 
mary health centers. The NHDSS has been used as a 
sampling frame for numerous studies in the fields of 
clinical research, epidemiology, health economics, and 
health-systems research. Nouna has a functional vital 
event registration system, which allows collecting data 
continuously on pregnancies, births, deaths, and migra- 
tion [17]. 

The VA questionnaire 

The Nouna questionnaire covers background character- 
istics of the deceased using structured filter questions 
on specific signs and symptoms experienced by the 
deceased up to the point of death. Additionally, a narra- 
tive section provides an opportunity to describe condi- 
tions not covered in the structured questions (see 
Additional file 1). Although the questionnaire is written 
in French, interviews with the HDSS population are per- 
formed by trained fieldworkers who translate the con- 
tent into local languages. The Dioula language is the 
most spoken local language, but several other languages 
are common, such as Bwamu, Moore, and Fulfulde. 

Verbal autopsy questionnaire data are collected every 
four to five months at the household level by 
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Figure 1 Map of NHDSS 



interviewers. They are then coded by physicians familiar 
with the 10 th revision of the WHO International Classi- 
fication of Diseases codes (ICD-10). 

We used the ICD-10 adopted in 1994 by the World 
Health Assembly. Its main use here is to classify causes 
of mortality as recorded at the registration of death. The 
ICD-10 also covers a conceptual framework of defini- 
tions, standards, and methods that have been closely 
linked and developed along with the classifications 
themselves. A restricted list based on ICD-10 has been 
used for the final physician coding (see Additional file 
2). 

Physicians' coding organization 

The VA coding sessions were organized locally by gath- 
ering 12 physicians working in the district hospital with 
an average working experience as general practitioners 
of four years. One of these physicians with detailed pub- 
lic health background guided the coding process. All 
physicians had good knowledge of patient management 
covering the areas of general medicine, care for pediatric 
inpatients, care for HIV patients, and basic gynecological 
and obstetrical care for women. Nevertheless, the panel 



sought opinion from external specialists in the area of 
interest when required. Based on the number of avail- 
able physicians, the panel consisted of three to four 
members. An agreement upon a given cause of death 
was only reached when two out of three members (66%) 
or three out of four members (75%) of the panel arrived 
at a consensus. Thus the panel coding process was more 
than majority-based and required that more than 50% of 
the panel members come up with the same cause of 
death. The cause was then ascribed to the final cause of 
death. The panel overwhelmingly agreed to classify the 
cause of death as undetermined if the available VA 
information did not allow them to make a final decision. 

Study design 

This study was designed as a comparative study using 
two methods of PCVA to ascertain causes of death 
respectively on two independent samples of VA ques- 
tionnaires collected in 2009 and 2010. 

The first sample, from 2009, was coded using the 
WHO-recommended method (Method 1). The second 
sample, from 2010, was coded using the extended 
method (Method 2). 
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Coding methods 

Method 1: As recommended by WHO [3], two experi- 
enced local physicians interpret the answers to the ques- 
tionnaire and independently determine the most 
probable cause of death. In the case of disagreement, a 
third physician is consulted. The cause of death is 
attributed only if supported by at least two physicians 
using ICD-10. 

Method 2: In 2010, Method 1 was extended using a 
panel of physicians in the case of a coding discordance 
between referee physicians. 

The VA coding procedure has been combined in a 
stepwise process shown in Figure 2. 

Verbal autopsy data collection 

Two key actors are generally involved in the process of 
VA data collection. Since the creation of Nouna HDSS, 
the event of death is registered in an active reporting 



system using community reporters, called community 
key informants (CKIs). Overall, 58 CKIs (one per village) 
report deaths occurring within households. Afterward, 
an assigned village interviewer collects information on 
the death. The trained field staff who visit households 
with a registered death have no medical background. As 
described above, they conduct the interview with the 
caregivers or relatives, translating the French VA ques- 
tionnaire into the local language. The interview usually 
takes place several months after the event with the per- 
son who assisted the deceased before the death. Figure 3 
presents the VA data collection flow chart in Nouna 
describing the interaction between fieldworkers and the 
community. 

Quality control 

Quality control is ensured by several checking mechan- 
isms put in place at different stages of the data 
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Figure 3 Verbal autopsy flowchart. The filing process steps are as follows: 1) Events processing for data entry; 2) Processing output of VA 
interviews; 3) Processing of VA forms for data entry; 4) Process followed in the case of problems reported on the VA forms; 5) Processing of VA 
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collection process. Whenever inconsistencies in col- 
lected information do not allow for a final diagnosis, a 
second interview is done by a field supervisor for consis- 
tency. Independently, the interview process at the 
household level is closely followed up by village supervi- 
sors in a random manner. At the data-entry level, atten- 
tion is given to the attributed codes to reduce errors of 
coding. 

Statistical method used 

The concordance rate was obtained for each method by 
taking the total number of VAs coded where there is 
agreement among physician coders over the total num- 
ber of VAs coded. 

The proportion test for two independent samples was 
applied to compare the proportions of undetermined 
cause of death achieved using the different methods. 

Results 

Verbal autopsy data coded 

Out of 1,256 deaths collected over the study period, 640 
were coded in 2009 using the first coding method 
(WHO), while 616 deaths were coded in 2010 using the 
locally-adapted method. 

Agreement between physician coders 

Out of 640 deaths coded in 2009 using the WHO 
method, there was an agreement on 315 diagnoses. This 



represents a concordance rate of 49.2% for the first 
method. Applying the same procedure to the VA 
records of 2010, agreement could be achieved for only 
219 records, resulting in a preliminary concordance rate 
of 35.6%. 

Involvement of the physician panel increased agree- 
ment on the final cause of death to 607 diagnoses. 
Thus, the latter method yielded a concordance rate of 
98.5% among physician coders, given the two stages of 
analysis. With additional involvement of a physician 
panel in the case of disagreement among the principal 
coders, the discrepancy among physician coders could 
thus substantially be reduced to less than 1.5%. 

The results of the proportion test showed that the 
proportions of undetermined causes of death achieved 
by the two methods were significantly different (p value 
< 0.0001). The study shows a significant reduction in 
the percentage of undefined causes of deaths. 

Predominant cause-specific mortality fraction 

Our findings indicate that malaria is the leading cause 
of death, 37.3% in 2009 and 37.9% in 2010, of total 
deaths registered (Figure 4). Here the undetermined 
CODs are not displayed, assuming that the non coded 
CODs follow the same pattern as the coded CODs. 
Malaria is followed by pneumonia and diarrheal dis- 
eases. Figure 4 shows similar patterns in deaths using 
the two methods. 
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Figure 4 Comparative cause-specific mortality fractions of Methods 1 and 2. 
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Discussion 

Our findings provide evidence that the choice of verbal 
autopsy coding method has a highly significant impact 
on the results of PCVA. It indicates that the improve- 
ment of an empirical method of PCVA, like the WHO- 
recommended method [3], through use of a physician 
panel in case of COD-coding disagreements, leads to a 
high reduction of the proportion of undetermined 
causes of death. Although this method of panel coding 
necessitates additional resources and time for physician 
coders as compared to the standard coding procedure 
described by Soleman et al [5], and especially to the sin- 
gle-coding procedure described by Joshi et al [13], it 
brings a large improvement in the existing methods 
determining the probable causes of death. 

The findings of Fottrell et al [9] support our results, as 
an initial agreement of 60% among two physicians was 
shown to increase to more than 80% when a review is 
done by one additional physician. Thus, our findings are 
in stark contrast to those of Joshi and colleagues who 
suggest reducing coding to one physician only [9]. We 
cannot exclude the fact that the quality of coding might 
depend on the VA questionnaire used per HDSS site, as 
discussed in several INDEPTH meetings over the past 
five years. The current VA questionnaire available 
through WHO/INDEPTH tries to overcome limitations 
of existing VA questionnaires, offering separate versions 
for different age groups and providing comparability 
over different countries. Thus, NHDSS has moved to 
the updated WHO/INDEPTH questionnaire in 2011. 
For coding, the Nouna site uses the restricted classifica- 
tion list suggested during the INDEPTH Meeting in 
Uganda in 2008, comparable with other HDSS sites. 

On one hand, our data possibly suggest that the new 
multicoding system of deaths doesn't necessarily affect 
the mortality pattern, although it results in changes in 
the proportion of deaths within the different groups of 
leading causes of death. Undeniably, the suggested 
method is more time-consuming and costly, but it is 
also more efficient. However, this is the first time that 
such a panel discussed the questionable cases. In sum- 
mary, the procedure might be especially helpful in 
HDSS sites where high rates of undetermined CODs are 
observed. 

The use of automated Bayesian models to assign the 
most likely causes of death tested by Byass [18] are cur- 
rently under investigation in the Nouna HDSS. The 
main gains achieved from this method are a reduction 
in time and cost needed to complete the coding process. 
Additionally, as the model doesn't involve different phy- 
sicians over time or in different countries, it aims to 
provide comparable results within HDSS sites over time 
and across different HDSS sites. However, its use is still 



limited to certain sites and the result comparison with 
PCVA approaches still shows some discrepancies in 
comparison to the PCVA results [12]. 

However, given that the computer-based probability 
approach to VA interpretation is designed to overcome 
the weakness of physicians' reviews, preliminary results 
are promising, but are not fully convincing [9-11]. At 
present, the main problem in choosing an optimal 
method of coding for VA is that no gold standard is 
available and comparison among various methods 
remains limited. Currently, both methods might profit 
from comparing their results with those by the other 
method. 

For resource-poor settings, a reliable and affordable 
method of VA coding remains a necessity, as mortality 
data remain important to guide decision-makers for 
health planning purposes. While waiting to scale up use 
of the computer-based model, the improved WHO 
method proposed in our study could be applied as an 
alternative method for coding, as it offers a good rate of 
concordance among physician coders. 

Despite verbal autopsy being a useful tool in deter- 
mining causes of death, the method has some limita- 
tions. Previous studies note these shortcomings in detail 
[2,5,9,13]. Because verbal autopsy is based on data col- 
lected through an interview process, and based on signs 
and symptoms exhibited, it is subject to recall bias and 
misreporting. Physicians have different experiences and 
knowledge in coding that could lead to different inter- 
pretations of the diagnosis [5,9]. 

While PCVA has some well-known limitations [4], the 
shortcomings of the tool are known and quantifiable. 
These deficiencies, however, should not prevent coun- 
tries requiring information on causes of death from ben- 
efiting from the use of VA when no practical alternative 
for obtaining these data exists. Few studies are available 
on the use of different physician coding methods that 
produce better results. However, the approaches for 
cause of death assignment most commonly used either a 
panel of expert physicians or two or more physician 
coders who independently review the data and arrive at 
a diagnosis [14,15]. Despite its acknowledged limitations 
[13], PCVA is still considered the best possible method 
to get cause of death estimates in areas where vital 
events registrations systems are limited or not available. 

Cohen's kappa as a measure of agreement couldn't be 
applied, as both samples used here were independent 
(years 2009 and 2010) and the coding was done indepen- 
dently in a blind manner by different physicians. Given 
this constraint, we focused only on the comparison of con- 
cordance rate between the two samples. This approach to 
analysis allowed us to attain a simple but effective measure 
for the improvement of the extended coding method. 
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There is a need for further study to confirm our find- 
ings in other settings. This will have the advantage of 
adopting a unique method for HDSS sites within the 
INDEPTH Network and to some extent to other sites 
outside the network interested in more accurate physi- 
cian-certified verbal autopsy coding methods. 

Conclusions 

Verbal autopsy remains essential to capture and deter- 
mine probable cause of death, especially in the context 
of low-income countries like Burkina Faso, where it is 
estimated that roughly 75% of deaths occur at home 
[19]. The VA process has the ability to contribute sub- 
stantially by informing policymakers on real mortality 
data and allowing countries to monitor trends toward 
attainment of Millennium Development Goals, in parti- 
cular those related to maternal and child health out- 
comes. The advantage of involving a physician panel in 
the coding process as suggested here is obvious, as it 
allows coding of an additional 50% of VAs. Importantly, 
this method promotes interactive discussions among 
physicians involved in the coding process, similar to 
what physicians are already doing during their clinical 
presentations on patients. Thus, the panel method pro- 
vides a framework for scientific discussion among physi- 
cians, allowing everyone to update their knowledge. 

Our study presents an alternative method of PCVA 
that substantially reduces the proportion of undeter- 
mined causes of death and therefore contributes to the 
death codification. We also aim to advocate for harmo- 
nization in the PCVA process, while encouraging the 
validation of the computer-based method of death 
coding. 
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Additional file 1: The VA questionnaire used in the NHDSS 

Additional file 2: The restricted cause of death list based on ICD-10 
used for the final physician coding 
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